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The detection of a completely known signal which may or may not be 
present in a finite sample of gaussian noise is considered from two points 
of view. The first examines the performance of a maximum likelihood de- 
tector operating on a finite set of discrete measurements of the stimidus as 
the set becomes large. The stimulus is either signal plus noise or noise alone. 
Examples are presented for signals in bandlimited noise, using as measure- 
ments either equispaced amplitude samples or derivatives at one instant in 
time. For both, the detectability grows without bound as the number of meas- 
urements is increased. The second point of view bases detection on a con- 
tinuous measurement {linear integral operator) which maximizes the de- 
tectability. Solutions have been obtained when the noise has a rational power 
spectral density. The detector utilizes a cross-correlation between stimulus 
and signal which is well known and a mechanism, designated extrapolation 
detection, which involves evaluation of derivatives of the stimulus. The con- 
tribution of the derivative measurements to the detectability is examined as 
the noise approaches bandlimited noise and is found in many cases to 
grow without bound. 

I. INTRODUCTION 

The problem under consideration here is the detection of a completely 
known signal which may or may not be present in a finite sample of 
gaussian noise. That is, we imagine a situation similar to Fig. 1 in which 
a stimulus is made up of either signal plus noise or noise alone and we 
ask, given T seconds of this stimulus, how accurately can we decide 
whether or not the signal is present. The noise is thought of as having 
been produced by a stochastic process and thus the question is really 
one of statistical hypothesis testing. 

This particular problem has been treated rather extensively, 1 " 7 and 
certain questions, even controversies, have arisen. These concern what 

423 



424 THE BELL SYSTEM TECHNICAL JOURNAL, MARCH 1961 



GAUSSIAN NOISE 
N(t) 



STIMULUS 



SIGNAL -I- NOISE 



CLOSED V (t) 

O TO T 



S(t) + N(t) 

Fig. 1 — Diagram of problem under consideration. 

constitutes a proper description for the stimulus, under what circum- 
stances can the stimulus be characterized by a finite number of samples, 
and under what conditions is perfect detectability obtained, i.e., when 
is it always possible to detect the presence or absence of the signal. 
Peterson, Birdsall and Fox have described the stimulus as being Fourier 
series bandlimited and by so doing have obtained quite different results 
from the other authors, who for the most part consider stationary gauss- 
ian noise. In many cases, finite-duration stimuli have been character- 
ized by a finite number of samples usually chosen so they are independ- 
ent, and maximum likelihood detectors operating on these samples have 
been developed. This has led to the equivalent of a correlation detection 
process in which the test statistic is the integral of the product of the 
stimulus and a function derived from the signal. Such detectors always 
produce finite detectability. On the other hand, Slepian 7 has pointed 
out by an argument involving analytic continuation that many signals 
can be perfectly separated from noise provided the noise is considered to 
have a bandlimited spectrum. Clearly some mechanism in addition to 
correlation detection is inherent in Slepian's result, and indeed he points 
out one such detector. 

The results of Peterson, Birdsall and Fox have been used extensively 
for comparison with the performance achieved by humans and other 
animals, and questions as to the validity of such comparisons originally 
motivated this investigation. However, it seems very doubtful if the 
mechanisms which will be developed can have anything to do with per- 
ception. In addition, we have chosen to work with stationary gaussian 
noise rather than Fourier series bandlimited noise, the former being a 
much more satisfactory characterization of real noise. 

Two different attempts to better understand the questions cited above 
have been undertaken. The first examines the performance of a maxi- 
mum likelihood detector operating on a finite set of discrete measure- 
ments of the stimulus as the set becomes very large. The results show 
cases where the detectability grows without bound. Thus, the charae- 
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terization of the stimulus by a finite set of measurements is incomplete. 
However, in some cases, a law of diminishing returns operates so that 
the rate of increase in detectability slows as the number of samples is 
increased. 

The second study bases detection on a continuous measurement 
(linear integral operator), which is the solution of an optimizing integral 
equation. The test statistic so obtained has two parts, one similar to 
correlation detection, the other based on measurements of the deriva- 
tives of the stimulus. The contribution of this latter term is usually the 
smaller of the two, but, where the noise spectrum approaches a band- 
limited form, it may grow without bound. In addition, it may be im- 
portant if the stimulus is very short. 

Both maximum likelihood detection with a finite number of samples 
and the integral equation for the continuous statistic have been pre- 
viously presented. The new contributions arise from the more complete 
solutions which have been obtained. The most significant result is un- 
doubtedly the solution of the integral equation in closed form so that 
its characteristics and particularly its asymptotic properties for many- 
pole noise can be seen. The derivative detector, which will be termed 
extrapolation detection, was apparent from this solution. 

II. DETECTION WITH A FINITE NUMBER OF SAMPLES 

In this section we will derive the maximum likelihood detector for de- 
tecting a known signal in gaussian noise from a finite number of samples 
of the stimulus and apply this detector to two specific problems involv- 
ing bandlimited noise. Each sample results from some linear operation 
on the stimulus and the samples need not be independent. The deriva- 
tion of the detection equation differs only slightly from previously pub- 
lished work, 5 and is included to lead clearly into the specific problems, 
which are the principal new results. In the problems the behavior of 
the detector is studied as the number of samples becomes large, first 
when the samples consist simply of amplitude measurements of the 
stimulus and second when the samples are a set of derivatives at one 
point in time. 

2.1 Maximum Likelihood Detector 
The stimulus 

l N(t) O^IIT, (I) 



Y{t) - \N(t) 4- 5(0 
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is either a gaussian noise N(t) or that noise plus a known signal S(t) 
and is observed for the interval ^ t ^ T. The n samples, 

Yi , Y 2 , • • ■ ,Y„ , 

on which' the detection is made are obtained by n linear operations 
L\ , L 2 , • • • ,L n on the stimulus 

Yi = L f [Y(t)] i= ],■■-, n. 

Because of their linearity, 

L { [N(t) + 5(0] = LAN(t)] + LAS(t)] = N< + S it 

and N, will be gaussian random variables which may be completely 
characterized by their matrix /3 of correlation coefficients, 

fa = E<NiNj>, 

and by their means which for simplicity will be assumed to be zero, 

E<Ni> = 0. 

The density function of the F, samples when the stimulus is noise 
alone may then be written 

MVi , ■ ■-,!/„) = (27r)-" /2 | T* exp { -\ £ ftTWil, 

where ?/i , • • • ,y„ are the dummy arguments of the density function 
corresponding to Y x , ■ ■ ■ ,Y n and | /S | is the determinant of 0, with all 
sums going over the range 1 to n unless especially indicated otherwise. 
The density function of Yi for signal plus noise is simply 

fssiyi , • • -,!/,,) ~ Jn(Vi - Si , • • •,>]„ — S„) 

because the signal is additive. Thus the likelihood ratio L(y x , • • -,y n ) is 

t („ . . „ \ - My* > •••»y») 

which when evaluated for these density functions becomes 

Uvi , • • -,2/n) = oxp {-i Z fa-'SiSi) exp lEtoi~ l S4lA. 

i.j i,j 

A maximum likelihood detector says that signal is present if test 
statistic L(Fi , •• ',Y n ) is greater than some threshold a and will maxi- 
mize the conditional probability of detecting a signal when it is present 
for a given conditional probability of indicating signal for noise alone. 
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However, L is a monotonic function of the statistic v, 

v = E e<r l SiYi , (2) 

it J 

and consequently an equally good test is <p > a c , where a c is an equiva- 
lent threshold. <p may be characterized by two density functions, one 
if the stimulus is noise alone, the other for signal plus noise. For noise 
alone, <p N (the subscript "N" designates noise alone, "SN" signal plus 
noise) is gaussian with zero mean and variance 

For signal plus noise <p S N is also gaussian with the same variance but 
with mean 

E<<p SN > = Z Pif^iSi . 

i.j 

The density functions for <p are pictured on Fig. 2. The 1 effectiveness of 
this detector as indicated by the signal detection probability at a given 
false alarm rate can be characterized by a single number d, which is the 
ratio of the squared mean of the signal plus noise distribution to the 
variance of either distribution. The larger d is, the more completely 
separated are the distributions on Fig. 2 and the higher will be the de- 
tection probability. This number d is then 

d = E/3o _1 ^S ; . (3) 



An alternate form for the statistic <p from that given in (2) is 



(4.. 



PROBABILITY DENSITY 
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Fig. 2 — Two density functions characterizing statistic f. 
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where the Z/s are solutions to the equations 

T,PiiZ i =S j j = l,--,n (5) 

and d may be expressed 

d = Z ZiSi • (6) 

i 

This form is usually preferable for computations since it involves the 
solution of n linear equations rather than the inversion of an n X n 
matrix. In addition this form more closely resembles the integrals which 
will appear when continuous statistics are considered. 

To summarize, a statistic v which operates on a set of n correlated 
samples and which is equivalent to a maximum likelihood statistic has 
been developed. Signal is indicated if <p is greater than some threshold. 
<p is formed as a linear sum of the samples, it has a gaussian distribution, 
and it has the same variance for both noise alone and signal plus noise 
cases. The performance of the detector may be characterized by a single 
number d = [E<<p S N>] 2 /E<<p N 2 >, the larger the d, the better the perform- 
ance. 

2.2 Detection of Sinusoid in Bandlimited Noise with Time Samples 

The argument presented by Slepian indicates that theoretically, be- 
cause of the analytic nature of the noise, a sinusoid can always be de- 
tected in spectral bandlimited noise. However, this result says nothing 
about how fast the detectability increases with the complexity of the 
detector. In this section an example is examined in which the stimulus 
is time sampled with n samples equally spaced over the interval ^ 
t ^ T and detectability is computed as a function of n. In addition to 
the general behavior of this function, it is of special interest to note 
whether any peculiarities occur at n = 2WT (the Nyquist rate), W 
being the noise bandwidth, since this is the maximum number of in- 
dependent samples which may be formed. The correlation function of 
the noise is 

B(r) = E<N«W(l + r)> = ^^, 

where the noise has unit mean square amplitude so the matrix of cor- 
relation coefficients /3„ can be written 

sin 7T— - j) 

fc = — n - 
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with 

n q = 2WT. 

Unfortunately, no analytic way for either inverting this matrix or solv- 
ing (5) is known, hence the detectability was computed numerically. 
This computation was carried out on an IBM 704 machine for a signal 
with frequency centered in the noise band 



o i ■ * ( n i • n i\ 



A being the amplitude and vnjln being an arbitrary phase chosen for 
computational convenience. The normalized results of a solution of (5) 
and (6) are presented on Fig. 3, where d/A 2 is given as a function of 
the number of samples n/n q and of the stimulus duration in terms of 
the number of independent samples n q . The curves exhibit a knee, not 
at n = n q but for n a bit larger than n q . Detectability continues to in- 
crease but the rate of increase becomes imperceptible. The curves are 
all carried out to a matrix of size 128 X 128, which is the limit of the 
capacity of the computer program. Double precision arithmetic and a 
sufficient error analysis were used to insure the accuracy of the results. 
The increase in detectability beyond n = v„ is essentially equivalent to 
that which would be obtained by increasing T to T 4- 2/W and sampling 
at the Nyquist rate. Heuristic-ally we can say that, by adding extra 
points inside the interval, it is quite easy to predict N(t) two independent 
sample times beyond each end of the interval, but very hard to predict 
further. In an unpublished proof Slepian has shown that the quadratic 
form for d given by (3) does become infinite for bandlimited noise as n 
becomes infinite. However, the present example indicates it increases 
at an exceedingly slow rate. Clearly a statistic which improves more 
rapidly is desirable, and such is evaluated in the next section. 

2.3 Detection of a Constant in Bandlimited Noise Using Derivatives 

The solution for the optimum integral operator detector carried out 
in the next section produced a statistic involving derivatives of the 
stimulus. This result suggests trying derivatives for bandlimited noise, 
particularly since all derivatives of a bandlimited stimulus exist. Con- 
sequently, the detectability achieved by n samples, which are the stim- 
ulus and its n — 1 derivatives evaluated at one point in time, is studied. 
This quantity, as will be seen, has the pleasant characteristics of being 
analytically rather than only numerically determinable and of increasing 
uniformly with n rather than exhibiting the knee curves of the time 
samples. A curious property is that the duration of the stimulus is no 
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Fig. 3 — Normalized results of a solution of (5) and (G), with d/A 2 as a function 
of number of samples n/n q and of stimulus duration in terms of number of inde- 
pendent samples n, . 

longer a factor in detectability since, theoretically at least, any number 
of derivatives can be measured from as short a sample as desired. 
Detectability can again be computed from (5) and (6), where 

P„ = E<N (r - l) (0)N (g - 1) (0)> 

is the correlation of the r — 1 and s — 1 derivatives, 



N W (t) m 



dt- 1 
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The correlation coefficient may be written 

M = J- f C?(co)(-iw) r ~ 1 0'w)'~ 1 d« > (7) 

27T • —ao 

where G(cj) is the power spectrum of the noise 

G(w) = f E<N(t)N(l + r)>e~ iuT dT. 

If bandlimited noise with a flat spectrum from —1 to +1 rad/second 
and unit rms amplitude is selected, then (7) yields 

\( - ^ ( - 1 ) |rH- if r + s is even 

n = V + s- 1/ 

1 if r + s is odd. 

A solution for (5) and (G) with these coefficients can be effected, since 
the determinants involved are reducible to a form with a solution attri- 
buted to Cauchy. The answer can probably be written on a large enough 
sheet of paper for signals having simple derivatives such as sinusoids, 
but the result is especially compact for a constant for which 

8(0) = K, S (B) (0) = £~£ =0 n = 2,3,- • • . 

dt"~ l 

The evaluation, carried out in Appendix A, yields for d 

' = 4 ,,. , (2 r )! m l- w 

L2-" ,-1 /n!(ra — 1)!J 
where 

ii 



m = 



for n even 

- 4- - for n odd. 
2 2 

The asymptotic behavior of d for large m can be seen by substituting 
Stirling's approximation 

a! » V2rexp {—a + (log a)(a 4- |)} 

for the factorials in (8), thus reducing it to 

d^^e m6 "' 2) m. 19) 

The approximation is within 2 per cent for in = 20. 
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Equations (8) and (9) exhibit the behavior of a statistic in which d 
increases linearly with the number of samples, each sample being a 
derivative. A similar behavior will be shown for rational noise where 
one term in the detectability depends linearly on the number of deriva- 
tives which exist and form part of the statistic. The bandlimited noise 
differs from the rational noise in that all its derivatives theoretically 
exist and the detectability can be made, at least theoretically, as good 
as desired by making m large enough. Obviously, in any practical case, 
the number of derivatives which can be estimated is limited. In addition, 
the characterization of the random process as gaussian undoubtedly 
fails for high enough derivatives. 

Equations (8) and (9) are derived only for a signal which is a con- 
stant. However, a similar dependence on m would probably occur for 
sinusoidal signals. 

The prominence of derivatives as an effective statistic for both band- 
limited and rational noise gives a possible indication why detectability 
based on equally spaced time samples increases so slowly. These, being 
uniformly distributed, give poor estimates of derivatives. A more effec- 
tive distribution might well be n q independent samples spaced uniformly 
over the interval and the rest of the samples clustered as closely as pos- 
sible about two points at each end of the interval. Such arrangement is 
suggested by statistics for the rational noise case. 

III. DETECTION WITH CONTINUOUS SAMPLING 

The preceding section discussed the detection of a known signal in 
bandlimited noise using a finite number of samples of the stimulus as a 
statistic. In this section we consider the detection of a known signal in 
gaussian noise using as the statistic a continuous measure of the stimu- 
lus over an interval T in length. The noise is now taken to have a ra- 
tional power spectral density ; that is, its power spectrum can be repre- 
sented at the ratio of two polynomials in w 2 . Such noise can be thought 
of as resulting from the passage of ideal white gaussian noise through a 
finite linear lumped-element filter, although it need not actually have 
been produced in this way. For the purposes of the analysis, it is con- 
venient to think of the situation as shown in Fig. 4. White gaussian 
noise is passed through a filter whose transfer function is H(s), (Laplace 
transform of its impulse response) and to this may or may not be added 
the known signal S(t). T seconds of the combination form the stimulus 
Y(t). The problem is to decide from an examination of the stimulus 
whether or not the signal was present. 
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Fig. 4 — Diagram of continuous sampling situation. 

The detection scheme in this case is essentially an extension of the 
finite sampling procedure. One asks for that linear integral operator 
which will extract from the stimulus a statistic giving the maximum 
detectability. Thus, the statistic is obtained from 



ip = [ Y(t)Z(t) dt, 
Jo 



10) 



where Z(t) is that function of time which maximizes the detectability. 
Because the noise is gaussian of zero mean and the signal (when present) 
is simply added to the noise, the statistic <p again has a gaussian proba- 
bility density function whose mean value is zero or not zero according 
to the absence or presence of the signal and whose variance is the same 
with or without the signal. Thus it is reasonable to again define the de- 
tectability measure d as 



d = 



[E(<p SN )} 2 
E(<p N 2 ) 



(11) 



The optimization problem is thus to find Z(t) which maximizes d or, 
that which is equivalent, to find Z(l) which minimizes E(<p N 2 ) while 
holding E(<psn) constant. This latter form is a straightforward calculus 
of variation problem and its solution, the details of which are omitted, 
leads to the following integral equation for Z(t): 



[ R(t - u)Z(u) du = S(t) ^ i ^ T, 

where R(t) is the autocorrelation function of the noise, 

B(t) = E[N(t)N(t+ r)]. 
When (12) is satisfied, the detectability can be written 

d = f Z(t)S{t) dt. 
Jo 



(12) 



(13) 
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The discussion up to this point has not required that the noise have a 
rational spectral density. Unfortunately, it does not appear possible to 
carry (13) any further without actually solving (12) for Z(t), and this 
has only been done in certain special cases. In particular, if the noise 
spectral density is the reciprocal of a polynomial, the solution for (12) 
.an be exhibited in some detail; and furthermore if the signal is a sine 
wave, an exponential, or a constant the detectability can be expressed 
in a surprisingly simple form. 

3.1 All-Pole Noise 

If the noise has a spectral density G(oi), 

G(ta) = f R{r)e~ im dr, 

which is rational and contains only poles (2JV in number), it can be 
written in the form 

QM rx — l Er- (14) 

a — agar -\- a 4 a> 4 — • ■ • ± ciinu 

Such a noise could have been produced by passing white noise of unit 
spectral density through a filter whose transfer function H(s) has N 
poles, 

H(s) = — — — — - = -^-^ , (15) 

6o + bis + 6 2 s 2 + • • • + b N s N P(s) 

and the poles can be placed in evidence by writing the denominator 
polynomial P(s) as 

P(s) = T.b k s k = 6jr(s-7i)(s-72),-",(s-7*), (16) 

fc=0 

where the 7's are (possibly) complex numbers giving the pole locations 
and each has a negative real part. In terms of H(s), the spectral density 
can be written 

G(u>) = I H(ju>) | 2 . 

Thus the noise can be described in a variety of ways — by the constants 
do , a 2 , • • • , a 2w , or the set b ti ,bi , • • • , 6 V , or the pole locations 71 , 72 , 
• ■ • , y N and one constant b N , or even the magnitude and phase of the 
transfer function H(s) for real frequencies. The particular set of param- 
eters to be used will be chosen to simplify the final answer. 
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One characteristic of JV-pole noise is that its first N — I derivatives 
exist, while the Nth and higher do not. Because of this it is clear that 
a necessary condition for finite detectability of a signal S(t) is that its 
first N — 1 derivatives be continuous in the interval to T. If this con- 
dition is not satisfied; that is, if among the N - 1 derivatives of S(t) a 
discontinuity occurs, then the detectability is infinite. This is clearly 
true, because one could simply differentiate the stimulus enough times 
to produce a step function in the interval and this could always be found 
by measuring the change in the differentiated stimulus just before and 
just after the time of the step. 

Using this JV-pole noise, it is possible to exhibit explicit solutions to 
(12) and (13). Unfortunately, strictly speaking, (12) does not have a 
solution unless S(t) and its derivatives up to order N - 1 satisfy a 
certain set of boundary conditions (boundaries at and T). If S(t) does 
not satisfy this set of boundary conditions, and in general for an arbi- 
trary signal it will not, then (12) has a formal solution if Z(t) includes 
delta functions and their derivatives to order N — 1 at the end points 
of the interval (approached from inside the interval). The details of 
this argument are presented in Appendix B, where it is shown that the 
solution to (12) is 

Z(t) = z c (t) + Z M (0 (0 + /M (<) (< - T)], 

Z c (l) - Ea»S (2W (<), 

fc-0 

where the superscript (n) indicates w-fold differentiation with respect 
to time, and the a's and /3's are given by 



a, = E ft*+itV*"°(0) 

i = 0,1,2,-..^ -1 (18) 

fit - Z (-ijV^i'^C). 



with 



Ui(t) = £ lh-$ (k) (t) and Unit) = £ (-l)%S w (t). 

A-=0 *-0 

When this Z(t) is substituted in (13), the detectability becomes 

d = [ T Z c (t)S(t) dt + E (-DW^O) + fi#*(T)\. (19) 

Jo '=0 
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Among the several other ways of writing d, one which is convenient 
is the following (partly operator notation) : 

d - I Ui(t) dt 

(20) 

+ E s (,) (o){[(-im(?>)p(-p) + Pi(-p)p(p)]s(t)\ t=0 , 

i'=0 

where 

Pi(x) = £ b k+ ix k - i 

h=i 

and p is the derivative operator d/dt. The derivatives of S{t) and U(t) 
at and T are to be interpreted as the limit of the value of the deriva- 
tives approached from inside the interval. 

The form of Z(t) in (17) is quite interesting. The first part contributes 
a function of time which is similar to the conventional cross-correlation 
result. One simply multiplies the stimulus by this function and integrates 
the product. In the second part, the delta functions, when used with (10) 
to form the statistic, represent evaluating the stimulus and its first 
N — 1 derivatives at the ends of the interval. The derivatives at the 
ends give information about the stimulus outside the interval. Essen- 
tially they allow prediction or estimation of the stimulus outside the in- 
terval, and this information is to be added to that from straight cross- 
correlation. As N becomes larger the noise spectrum drops off faster at 
high frequencies and more derivatives of the stimulus are used (more 
derivatives of the noise exist); effectively, the stimulus can be predicted 
further outside the interval. Usually, this will mean that the signal can 
be detected better (see examples below). 

3.2 Damped Sinusoidal Signal 

As a particular example, consider the case in which the signal is a 
damped sine wave of arbitrary phase, 

S(t) = Ae~ al sin (cot + <t>) = Ae u + iV*', (21) 

where 

A = —. e 3 * and X = —a-\-ju. 

Since the detectability is of primary interest, specific values for the co- 
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efficients of the delta functions will not be calculated. The details of the 
calculations are carried out in Appendix C, where it is shown that 



...wi-ay-*-"' 



[• 



2X 
+ 21 A 



j |2 r iP(A)iv x+x - )r -iP(-x)f i 
'L a + a* J* 



(22) 



3.3 Exponential Signal 
For an exponential signal, 

S(t) = Ae~ at 
and the detectability from (22) becomes 

d = ^[P 2 (a) - P\- a ) e - 2aT ]. (23) 

With given signal parameters and noise filter, specific values of detect- 
ability can be calculated from this expression. 

As the number of poles in the noise filter increases, P (—a)/P (a) — » 0, 
assuming the poles are bounded away from the imaginary axis and that 
a > 0. In this case d becomes 

d-^A 2 P 2 (a)/2a. 

If as the number of poles is increased the dc gain of the filter is kept 
constant (or allowed to increase), then P 2 (a) increases without bound. 
This can be seen by thinking of P(a) in factored form, which for con- 
stant nc gain looks like 



p(«) = b n 



a — 7, 



and noting that | (a - t.)/t. I > 1. Thus, for fixed signal, more poles 
mean more detectability. A similar result obtains if a < 0. 

A noise filter of particular interest is a Butterworth filter, that is, one 
whose poles are uniformly distributed on a semicircle in the left-half 
plane. Such a filter gives noise whose spectrum is maximally flat low- 
pass and approaches ideal bandlimited noise as the number of poles 
increases. In this case, the approximate behavior of d for large JV can 
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be calculated by taking the poles as smeared out on a semicircle of radius 
a>o . Thus, 

Oo TT Jo L \ w 0/ Wo J 

and, therefore, 



dg* 



2aG(0) 



B\ 



(24) 



where 



B = 



exp \-l In 1 + ( - J + 2 - cos * rf*> . 
[t Jo [_ \coo/ wo J J 



A sketch of B versus a/w is shown in Fig. 5. Clearly B is greater than 
one and the detectability grows exponentially for large AT. 
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3.4 Sinusoidal Signal 

For an undamped sine wave (a = 0), (22) can be put in a more con- 
venient form by using the magnitude and phase of the noise filter trans- 
fer function, H(s), which can be written 

H(jco) = VG&)e~ mu) . 

The angle 0(u) then is the phase lag of the noise filter, a function of 
frequency. In these terms (22) becomes 

, A' 1 [rp,^ x sin (2coT + + 4>) + sin 2(0 - g) ( , 

2G(u) 2w 



where 

= (70/(/o). 

If wT » 1, that is, if the time is long so that there are many cycles 
of the sine wave in the interval, then the last term in (25) can be neg- 
lected. In conventional circuit analysis, is generally considered the 
time delay of a network; thus, the detectability includes a term pro- 
portional to twice the time delay of the noise filter. Roughly, this says 
that the derivatives at the ends of the interval allow extension of the 
stimulus n distance equal to the time delay outside each end. 

It is clear that the term grows without bound as the number of 
poles bounded away from the imaginary axis is increased. In the par- 
ticular case of noise with a maximally flat spectrum [Butterworth H(s)], 
this growth can be shown more explicitly. The contribution to from a 
single pair of poles located at — «oe ±J is 

2 (X 2 + 1) cos/3 = w 

o7 X 4 + 1 + 2X 2 cos 2/3 ' wo * 

To add up the contributions from .V poles on a semicircle would lead to 
a rather complicated expression, but an approximation for large N can 
be obtained by imagining the poles smeared out on the semicircle, so 
that the sum can be evaluated as an integral. Then 



2 jv f' ri 1 + X J cos fldp _2N ln 



CO + OlQ 
O) — O>0 



(20) 

O>0 1 + X 4 + 2X 2 COS 2/3 7TO) O) — O)o 

This shows clearly that, for large N, increases directly in proportion 
to N. As a sidelight, the proportionality constant, plotted in Fig. (i, is 
larger if the signal frequency is near the band edge. The apparent in- 
finity for o) = o)o is a mathematical fiction; it resulted from smearing the 
poles. For any finite N, is finite; thus, the curve in Fig. G really should 
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Fig. 6 — Proportionality constant. 

be rounded over at the peak. For signal frequencies outside of the noise 
band, the de testability becomes large simply because the 1/C?(a>) term 
multiplying everything in (25) becomes large. Even straight cross- 
correlation would give large detectability here. 

3.5 Constant Signal 

For a constant signal, S(t) = A, the detectability can be written (see 
Appendix C) 

d = m[ T - 

Note that the minus sign does not imply negative detectability; the 7's 
have negative real parts and so their sum will be negative. Equation (27) 
shows clearly that the detectability increases as the number of poles 
bounded away from the imaginary axis is increased. 

For iV-pole Butterworth noise of bandwidth io , (27) becomes (ex- 
actly) 



£f|. 

t-i 7* J 



(27) 



" = m[ T 



coo sin (w/2N) 



]■ 



DETECTABILITY OF SIGNALS IN GAUSSIAN NOISE 441 

which for large N becomes 

Here again the detectability grows directly in proportion to N for 
large N. 

IV. CONCLUSIONS 

We have presented solutions to some problems involving detection of 
the presence of known signals in gaussian noise. Thus, we are concerned 
with what a statistician would term hypothesis testing. Two general 
classes of detectors are studied, the first a maximum likelihood detector 
operating on a finite number of samples of the stimulus, the second an 
optimum integral operator treating the stimulus as a continuous func- 
tion. However, the new results lie not in the general detection equa- 
tions, which differ little from ones previously given, but rather in the 
specific solutions to these equations. 

In the finite sampling case, detectability of a sinusoid or constant in 
bandlimited noise is computed for the cases where the samples are 
equally spaced time samples spread over a finite duration and where the 
samples are measurements of successive derivatives at one point in time. 
As the number of samples increases, detectability increases without 
bound for both cases. However, for the time samples the rate of increase 
is very slow for a large number of samples while for derivatives the rate 
becomes a linear function of the number of samples. 

For optimum linear integral detection a general solution is presented 
for arbitrary signals in noise with a rational all-pole spectrum. The solu- 
tion in closed form is sufficiently tractable so that the asymptotic be- 
havior of certain simple signals can be evaluated as the number of poles 
in the noise becomes very large. The solution puts in evidence two differ- 
ent detection mechanisms, one involving integration of the product of 
the stimulus with a function derived from the signal, the other involving 
measurement of the derivatives of the stimulus. The first is denoted 
correlation detection, the second extrapolation detection. Usually, the 
term arising from correlation detection is the more important. However, 
if the stimulus is very short or if the noise spectrum has a great number 
of poles, the extrapolation term may become relatively large. For signals 
such as a sinusoid it grows without bound as the number of poles in- 
creases. 

What are the implications of these solutions on previous detection 
results? Probably they have very little bearing on the perception prob- 
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lem which engendered the study, since it seems unlikely that animal 
sense organs embody the mechanisms implied by the solutions or that 
the characterization of exactly known signals in gaussian noise is appro- 
priate. Both the solutions and the character of the stimuli differ signifi- 
cantly from the Fourier series bandlimited case treated by Peterson, 
Birdsall and Fox. In particular, the extrapolation detection does not 
appear in their universe. Also, we feel that the characterization of the 
noise as described by a correlation function is, to say the least, more 
suited to the present style of engineering and, to say the most, a much 
more satisfactory model of most detection situations. 

The practical impact, if any, of the detectors developed here would 
seem to inhere in situations where short pieces of valuable signals must 
be detected and a great quantity of computing equipment is available. 
Such might be the case for some space communication problems. 

A number of unsolved problems arise directly from the work. For a 
finite number of time samples of the stimulus, the optimum distribution 
in time of these samples is unknown. Spectra with zeros as Avell as poles 
have not been treated with anything near the elegance of the pure pole 
situation. Only very specific classes of signals have been studied. It 
would be of interest to establish which signals give unbounded and which 
give bounded detectability as the number of poles in the noise increases. 
Finally, only the case of signals known exactly has been examined. The 
far more difficult area involving signals with random parameters is al- 
most untouched so far as practical solutions are concerned. 
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APPENDIX A 

Detection in Flat Bandlimited Noise by Estimating Derivatives 

In the main body of the paper it was shown that, for samples which 
are derivatives, detectability in terms of d can be determined from (5) 
and (6) 

TtfaZi = Si, j = \, ... ,n (5) 

d = ^ZZjSj. fo) 
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Sj is the j — 1 derivative of the signal evaluated at t = and /3 tJ , the 
correlation coefficient of the noise derivatives, is 



firs = 



( ! J ( - 1 )' r+ » s if r + s is even 

\r + s - 1/ 







if r + s is odd 



for flat bandlimited noise with unit rms amplitude. 

Equation (5) may be written out in matrix form for odd n as 




=Fl/n 



±]/n 




z, 




Si 







Zo 




s 2 






z* 


= 


s 3 


l/(2n - 1)_ 




Zn. 




_s n _ 



1 -1/3 

1/3 -1/5 

-1/3 1/5 

L±l/n 



and a similar form for even n. 

This equation may be simplified by separating into two equations and 
multiplying by minus one in appropriate places to remove minus signs. 
Two forms occur, one for even n, the other for odd n. For n odd, 



' 1 1/3 1/5 

1/3 1/5 1/7 

1/5 1/7 1/9 

[_l/n • • • 



1/n 




Zil 
-Z 3 




Si] 






z & 


= 


s 8 


l/(2n - 1)_ 




_±z n _ 




_±5„_ 



(28) 



and 



1/3 1/5 1/7 

1/5 1/7 1/9 

1/7 1/9 • • ■ 

\/n ■ ■ • 



1/n 




Z 2 I 

-z* 




•So "1 

-Si 






z 6 


= 


s 6 


l/(2n - 3)_ 




_±z„_l_ 




_±s„_,_ 



(29) 
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1/(71 - 


- 1) 




z, 
-z 3 




A 

-s 3 








z b 


= 


s b 


(2n - 


"3)_ 




±Z n -l 




_±S n _i_ 



(30) 



For n even, 

1 1/3 1/5 • 

1/3 1/5 1/7 • 

1/5 1/7 1/9 ■ 

Ll/(n - 1) 

and 

1/3 1/5 1/7 •■ 
1/5 1/7 1/9 ■■ 
1/7 1/9 • • • 

|_l/(n+ 1) 

The determinants of these matrices can be evaluated by applying a rule 
attributed to Cauchy. In general, the rule says that a determinant whose 
ijth. element is 



l/(w+ 1) 




zi 
-z A 




-s< 






z. 


= 


& 


l/(2n - 1)_ 




_±z n _ 




_±s n _ 



• (31) 



Ma = 



^ + bj 



has the value 



at + bj 



II II (a f - a y ) (6.-- 6/ 

n fr («.- + w 



For the particular cases considered here, a, and 6y have especially simple 
forms. For example, for (28), a* = 2t — 1 and 6y = 2j — 2. 

In addition, all cofactors of the matrices are also of Cauchy form. 
Hence, it is possible to invert the matrices by the method of cofactors 
and thus solve the equations. Such solutions are quite complex for arbi- 
trary signals. However, an especially simple answer can be obtained for 
a constant since 

8 t = K, 

Si = iVl, 
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where K is the signal amplitude. Equation (6) reduces to 

d = Mi = Z X K. 

Z\ may be determined by the well-known method for solving equations 
as the ratio of two determinants, 



Zi = 



K 


1/3 


1/5 


l/(2m - 


1) 





1/5 











l/(2m + l) 




l/(4w - 


3) 


1 


1/3 


1/5 


l/(2w - 


1) 


1/3 


1/5 








l/(2m - 


1) 




l/(4w - 


3) 



where 



m = 



n + 1 



for n odd 
for n even. 



Application of Cauchy's rule and the solution for d yields 

^f (2m)! f 

|_2 2 "<- 1 m!(m-l)!_ ' 

which is the result utilized in the main part of the paper. 

APPENDIX B 

In this appendix we give a general solution to the integral equation 



[ R(t - u)Z(u) du = S(t) ^ t ^ T, 

Jo 



(32) 



where R(t) is the correlation function of a noise whose spectral density 
is a rational function of frequency having only poles and S(t) is an 
arbitrary known signal. The solution of the equation can be expressed 
in a number of different forms. The particular one developed here has 
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the great advantage c I-eing an explicit function of R(t) and S(t) ratber 
than involving the sot. + jod of a set of linear equations. In addition, it 
possesses the aesthetically pleasing property of not involving analytic 
continuation of S(t) out ,',de the interval ^ t ^ T. The noise spectral 
density can be written 



G(«) = ! 



Q(P) 



Q(p) = E (hkV k . (33) 



If we think of Q(p) as an operator with p interpreted as d/dt we see 
that 

Q(p)[R(t)} = Q(p) f °° *, «*" p. = 5(0, (34) 

where 8(t) is the Dirac delta function. Operating formally on both sides 
of (32) with Q(p) yields 



N 

I 



Z c (t) = Q(p)[S(t)] = Ea tt S l2B (0, < t < T. (35) 



The subscript has been added to Z to indicate that this may be only part 
of the answer and the superscript (n) indicates w-fold differentiation 
with respect to time. If (32) had a Z(t) solution which was continuous, 
then (35) would be that solution. But the fact that (35) is continuous 
(as it would be if S(t) and its derivatives were continuous) does not 
prove that it is the complete solution. In fact, one can readily verify that 
(35) is not the complete solution by inserting it back in (32) and seeing 
if (32) is satisfied. It turns out that (35) is indeed part of the answer, 
and the remaining part is found by just this process of inserting (35) 
back in (32) and finding what is missing. If we imagine for the moment 
that S(t) is extended in some arbitrary way outside the interval (so 
that it is Fourier transformable and the function and its derivatives go 
to zero at ± °o ), we can write 



/ R(t — u)Zc(u) du — 



\ C" - £, - C] [R{t ~ u)Zc{u) du] - 



(36) 



The first integral on the right is a normal convolution of Z c and R, and 
if Z c from (35) is substituted in Ave get back exactly S(t). The second 
and third integrals are evaluated by repeated partial integration, or, 
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what is equivalent, by finding an exact differential expression for the 
integrand. We first note that Q(p) can always lie' factored, 



Q(p) = P(P)P(-P), P(P) - Ztf, (37) 

A-=0 

where P(p) contains only left-half plane zeros. Now define 
U x (t) = P(p)[S(t)}= tw°*(i) 

k— a 
and (38) 

17,(0 = P(-p)[S(t)] = E(-1)%S (W «). 

A-=0 

The exact differential that we need is obtained by clairvoyance. It is 

^fE^ ,H, (^ (H '('-«) 

du j-i i-j 

N 



= Y,WW k \u)R(t - u) - U 2 (u)R {k \l - u)\ (39] 

fc=0 

N 

= Z c (u)R(t - u) - U s (u) Y,b k R (k) (t - u). 



fe-a 



Now, since P(p) has the left-half plane zeros of Q(p), the Fourier trans- 
form of P(p) [ft (01 will have only right-half plane poles and thus 

N 

P(p)[R(i)} = Y,b k R lk \t) = for t > 0. 

*-=o 

Therefore, when Ave use (39) in the middle integral on the right of (36), 
we get 

f ° B(e - u)Z c (tt) du = Z Z 6, +1 f/ 2 a '- , ' ) (0)« <, ' ) (t). (40) 

The third integral on the right of (36) is evaluated in a similar way, 
using now (39) with Us replaced by f/i and h by ( — l) k b k . , and noting 
that ]>(-p)R(t) = for / < 0. In this way we get 

f R(t - u)z e {u) du = Z £ (-i)V^ ,M) (r)B ,,, « - r). (4i) 

It is interesting to note that (40) and (41) depend only on values of 
S(t) inside the interval ^ t ^ T, so that the way in which S(t) was 
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extended outside the interval does not matter. To summarize this, we 
find 



/ R(t - u)Z e (u) du = 

Ja 

s(t) - E E [b k+1 u 2 {k - i] (o)R (i) (t) 



■ T 

'0 

(42) 



AT-l AT— 1 

EE 

=0 k=i 

(*-*>/m\I»W, 



+ (-l)Vi^i (W fl « - T)]. 

It is now clear that, for Z c to be the complete solution to (32), the double 
sum in (42) must be zero for all t in the interval. This is equivalent to 
the following boundary conditions on S(t): 

1^17^(0) = 

i = 0,l, ---AN - 1). (43) 



Af-l 



E(-i)Vi^i a '" l ' ) (7 7 ) = o 



If the signal is such that these conditions are not satisfied, then (32) 
has a solution only if Z(t) includes delta functions and their derivatives, 
that is 



N—l 

E 

i—D 



Z(t) = Z c (t) + E W (0 (0 + Pi8 (i) (t - T)]. (44) 



If this is used in (32), the delta functions bring out R and its derivatives 
evaluated at t and t — T, and the a's and /3's can be directly identified 
as 

ft- E(-i)Vif/i <t_, ' , (r). 

The detectability for a Z which satisfies (32) is 

d = I S(t)Z(t) dt; 
Jo 

thus, using (44), 

d = fs(t)ZM dt + E (-1)W°(0) + fiiS w (T)]. (46) 

Jo «'=0 



DETECTABILITY OF SIGNALS IN GAUSSIAN NOISE 449 

This can be put in a slightly different form which may be more con- 
venient by again partially integrating. Using another exact differential 
obtained by clairvoyance, which is 






d 

dt =o *= 



= -S(t)Z c (t) + 0/(0 (47) 



and observing that when this is inserted in (46) the terms evaluated at 
T cancel, we get 

d = I Ui\t) dt 

N1 « (48) 

+ E (-i)'s (0 (o) Zb k+ Au 2 (k - i] (o) + (-iW*-°(o)] 

i=0 fc~ » 

or, equivalently, in an operator notation, 

d = [ Ui(t) dt 

Jo 



N—l 

t 

Ml 

where 



(49) 

\ E s w) (o)[(-i)'Pi(p)^.(o + Pi(-p)^i(ou, 



#— i 
£ 



P«(«) - E fc+is'"'- 



In this form the summation only involves derivatives at / = 0, which 
in some cases simplifies the algebra of a solution. 

appendix c 

As a particular example, we calculate the detectability d for the case 
in which the signal is an exponentially damped sine wave, 

S(t) = Ae- at sin (<d + *) - Ae u + IV 1 , (50) 

where 

.4 = — . e 1 * and X = —a + j<a 
and the asterisk denotes complex conjugate. Using this in the expression 
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for detectability (20) or (49), we find that the second term — call it 
da — becomes 

d A = ES w (0){[(-l)7Mp)P(-p) + P.(-p)P(p)]-S(01w 

i=0 

= 2 Re E{I*[(-X)'P<(X)P(-X) + X'7\(-X)/'(X)] (51) 



i=n 



+ | A | 2 [(-X*)\P,(X)P(-X) + X* l P,-(-X)P(X)]}. 

The notation Re means "real part of." From the definition of Pi(x), 
one can readily verify that 

SW)- P( ' ) - f( ' ) , (52) 

,=o a: - y 

and this allows (51 ) to be greatly simplified: 

n P\\) - P\-\)~\ 



d d = 2 Re A 






2X 

(53) 



+ 2 A 



( ,|P(X)| 2 - |P(-X)| 2 



X 4- X* 
The first term in the detectability, (20) or (49) is simply an integral, 

[ T U?{t) dt = f [AP(\)e M + A*P(\*)e x ' l fdt 
Jo Jo 



= 2 Re 



(Ml) 

2\T ,-1 (X+X*)T 1 V ' 

i 2 P 2 (x)^ r iJ + 2MP(x) 12 ' 



X + X* ' 
Combining (53) and (54), we get 

> 2 (X) e 2Xr -P 2 (-X)1 



d = 2 Re 



[r* 



2X J 

C5i) 

+ 2 | ir [ im>|V— -,P(-x)r ] 



which is the general solution for any damped sinusoid. 

Three special cases are now considered, the pure exponential, the 
pure sine wave, and a constant (nc) signal. For a pure exponential 
signal, X — > — a where a is real in (55), giving 

d = f[P*(a) - P\-a)e- laT ]. (56) 

la 
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For a pure sine wave signal, X — > jo. The second term in (55) requires 
a little special treatment, but it is easily shown that 

| POO |V x+x,)r - \P(-\) | 2 

X + X* cr-o' 

T I P( fa) I 2 + P( ~ J ' a,) rfP( ^ } - £i^ dP(-i«) 

Now, P(jo)) is simply the reciprocal of the transfer function of the noise 
filter at the frequency w; that is, 

J'fl(u) 

P(jo) = l/II(jo) = 



VG(co) ' 
where 0(co) is the phase lag of the noise filter. Using this expression, 

Aj_ \ T + 26M _ 1° jOg + » + «)+ sin 2(» - ») -! (57) 



i = 



2G(< 

where 5 = d6/du. 

For a constant signal we can simply take (5G) and let a — > 0, which 
gives 



d = A 



\ T P\0)+ d -^\ I 
L da |o=oj 

4[ T + 2 a = 5T5)[ r - 2 S 1 /- 



(58) 
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